Comparison of the Complex Valued and Real Valued Neural Networks Trained with Gradient Descent and Random Search Algorithms
نویسندگان
چکیده
Complex Valued Neural Network is one of the open topics in the machine learning society. In this paper we will try to go through the problems of the complex valued neural networks gradients computations by combining the global and local optimization algorithms. The outcome of the current research is the combined global-local algorithm for training the complex valued feed forward neural network which is appropriate for the considered chaotic problem. 1 The Differences between Feed-Forward Real Valued and Complex Valued Neural Networks In the following paper we briefly introduce Real Valued Neural Network (further RVNN) structure [1] which consists of the neurons, where the last one can be described with the following equation (see eq.(1)):
منابع مشابه
Natural Gradient Descent for Training Stochastic Complex-Valued Neural Networks
In this paper, the natural gradient descent method for the multilayer stochastic complex-valued neural networks is considered, and the natural gradient is given for a single stochastic complex-valued neuron as an example. Since the space of the learnable parameters of stochastic complex-valued neural networks is not the Euclidean space but a curved manifold, the complex-valued natural gradient ...
متن کاملHandwritten Character Recognition using Modified Gradient Descent Technique of Neural Networks and Representation of Conjugate Descent for Training Patterns
The purpose of this study is to analyze the performance of Back propagation algorithm with changing training patterns and the second momentum term in feed forward neural networks. This analysis is conducted on 250 different words of three small letters from the English alphabet. These words are presented to two vertical segmentation programs which are designed in MATLAB and based on portions (1...
متن کاملEnsemble of single-layered complex-valued neural networks for classification tasks
This paper presents ensemble approaches in single-layered complex-valued neural network (CVNN) to solve real-valued classification problems. Each component CVNN of an ensemble uses a recently proposed activation function for its complex-valued neurons (CVNs). A gradient-descent based learning algorithm was used to train the component CVNNs. We applied two ensemble methods, negative correlation ...
متن کاملProcessing Symbolic Sequences by Recurrent Neural Networks Trained by Kalman Filter-Based Algorithms
Kalman filter (KF)-based techniques used for recurrent neural networks (RNNs) training on real-valued time series have already shown their potential. On the other hand gradient descent approaches such as back-propagation through time (BPTT) or real-time recurrent learning (RTRL) algorithms are still widely used by researchers working with symbolic sequences. The aim of this work is to show how ...
متن کاملLearning Polynomials with Neural Networks
We study the effectiveness of learning low degree polynomials using neural networks by the gradient descent method. While neural networks have been shown to have great expressive power, and gradient descent has been widely used in practice for learning neural networks, few theoretical guarantees are known for such methods. In particular, it is well known that gradient descent can get stuck at l...
متن کامل